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Abstract 

Researchers have examined the use of homemade PowerPoint games as an instructional 
technique to improve learning outcomes. However, test data have shown no significant 
difference in performance between high school chemistry students who created games and 
students who did not (Siko, Barbour, & Toker, 2011). One of the justifications for the use of the 
games is that students will, with practice, write higher-order questions when constructing the 
games. Two subject matter experts rated over 2,500 questions from games created by students in 
an environmental chemistry class through thematic analysis using Bloom’s taxonomy as a coding 
scheme. The students wrote primarily recall questions, and students who created games on two 
occasions did not write more questions than students who only created games once. This 
suggests that changes to the question-writing aspect of the game project may be necessary in 
order to see improvements in achievement when compared to control groups. 

Keywords: constructionism, homemade PowerPoint games, question writing, game design 


Resume 

Les chercheurs ont etudie l’usage de jeux PowerPoint maison en tant que technique pedagogique 
visant a ameliorer les resultats d'apprentissage. Les donnees des tests n'ont toutefois revele 
aucune difference significative quant au niveau de performance des etudiants du secondaire en 
chimie ayant cree des jeux et celui des etudiants n’en ayant pas con§us (Siko, Barbour, et Toker, 
2011). L’utilisation des jeux est notamment justifiee par l’idee que, pratique aidant, les etudiants 
ecrivent des questions plus complexes lorsqu’ils elaborent des jeux. Au moyen d’une analyse 
thematique utilisant la taxonomie de Bloom comme systeme de codification, deux experts 
chimistes ont evalue plus de 2500 questions provenant de jeux con§us par des etudiants d’une 
classe de chimie environnementale. Les etudiants ont redige principalement des questions faisant 
appel a la memoire, et les etudiants ayant cree des jeux a deux reprises n’ont pas produit 
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davantage de questions que ceux qui en etaient a leur premiere experience de creation de jeux. II 
appert que des changements a la redaction des questions sont requis au projet de jeux pour 
pouvoir ameliorer la performance par rapport aux groupes temoins. 

Mots-cles: constructionnisme, jeu PowerPoint maison, elaboration de questions, conception de 
jeux 

Introduction 

Designing games can be a difficult endeavor, and designing educational games has the added 
layer of including educational objectives in the design (Hirumi, Appelman, Rieber, & Van Eck, 
2010). In order for a game to be considered good, it should have an enticing storyline and keep 
the player motivated by providing the appropriate amount of difficulty (Rieber, Barbour, 

Thomas, & Rauscher, 2008). Adding to the complexity of using game design as an instructional 
tool is the programming software itself. Teachers are faced with not only teaching content but 
teaching computer science as well (Barbour, Thomas, Rauscher, & Rieber, 2010). However, 
there are several “low-tech” ways to have students design games using more common computer 
applications such as Microsoft PowerPoint. One line of inquiry into low-tech game design is 
homemade PowerPoint games from a template. 

Researchers examining the use homemade PowerPoint games as an instructional strategy have 
listed three philosophical justifications for their use: constructionist pedagogy, writing across the 
curriculum, and student-generated questioning strategies (Barbour, Rieber, Thomas, & Rauscher, 
2009). However, the use of homemade PowerPoint games has not been shown to increase 
performance on assessments when compared to groups who do not create games and were 
exposed to more traditional methods of instruction (Barbour, Clesson, & Adams, 2011; Barbour, 
Kinsella, & Rieber, 2011; Parker, 2004; Siko, Barbour, & Toker, 2011). These findings have led 
researchers to examine these justifications in greater detail. In particular, researchers have 
questioned whether the construction of games does indeed make students write more higher- 
order questions (Barbour, Kromrei, et al., 2009). 

In this paper, I describe homemade PowerPoint games and detail the implementation of the 
project in the classroom. I follow this with a review of the literature supporting the philosophical 
justifications for their use. After reviewing studies involving homemade PowerPoint games, I 
will share the findings of my question analysis from a recent study involving the use of these 
games to teach chemistry. Finally, I will discuss potential for future research and provide 
recommendations for practitioners who wish to use the games as an instructional tool. 

Literature Review 

A homemade PowerPoint game can be any game created by students using MS PowerPoint. 
However, templates that can help students by providing structure to their games can be found at 
http://it.coe.uga.edu/wwild/pptgames/PPTgame-templatel.ppt (all of the studies reviewed in this 
paper have used this template). The template includes an introductory slide as well as slides for 
the game narrative, directions, and objectives. The rest of the game, including the question slides 
and linkages between those slides (i.e., via the action button feature in MS PowerPoint) is 
constructed by the students. If the students use the template, the game begins with an 
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introductory slide that directs players to information about the game and to the starting point for 
the game, an example of which is shown in Figure 1. 



Figure 1. A typical introductory slide to a homemade PowerPoint game 

Students generate a narrative slide, which provides the story behind the game. Students also 
generate slides that tell players how to play the game and how the game ends (i.e., how a player 
wins the game). Examples of these are shown in Figure 2 and Figure 3. In this game, an external 
game board is used in conjunction with the PowerPoint file. 


The Story of the “what about air?” 

One day Lola was flying to Florida on her private 
jet. As she gazes out the window, she begins to 
wonder... Lola began wondering all about the air 
and how everything worked around the earth. Lola 
continually asks the pilot all of the questions she 
begins to think of. As you play this game you (the 
pilot) will help Lola answer all of her questions 
about the air and the earth! When you give Lola a 
correct answer, the plane will continue on its way to 
Florida... but if you fail to answer her question 
correctly the pilot will get off track and start going in 
the wrong direction! Good luck, and try your best to 
help Lola get to Florida. Home Page 


Figure 2. A narrative slide for a typical homemade PowerPoint game. 
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Game Directions 

• The goal of the game is to get your pawn (Lola) from start 
to home (Florida). 

• To play the game you have to pick a pawn (which is 
representing Lola) roll the dice to see who goes first. Then 
who ever wins the roll gets to pick a number on the 
question board. Answer the question that shows up and if 
you get it correct you get to roll the dice and move your 
pawn (Lola) the number shown on the dice. If you get one 
wrong SORRY you do not get to move closer to your 
destination. 

• To win the game you have to move your pawn (Lola) all 
the way around your board to home (Florida). 


Return 


Figure 3. A homemade PowerPoint game slide that describes how to play the game. 

The game itself is usually played by answering a series of multiple choice questions related to 
some content. Figure 4 is an example of a question related to the direct relationship between 
volume and absolute temperature. 


A gas has a volume of 10m 3 at a standard 
temperature 273K. What volume will the 
gas occupy if the Kelvin temperature is 
doubled? 



* / 


Figure 4. An example of a question in a homemade PowerPoint game. 

For this question, the correct answer is 20m . Clicking on this answer would direct the student 
playing the game to another slide indicating that the player is correct and instructing the player to 
proceed. Clicking on the other answer would have a negative consequence (e.g., loss of turn, 
moving backwards, etc.). 

There are three justifications for using homemade PowerPoint games as an instructional tool: 
constructionism, microtheme writing, and question writing. First, the design of games is 
consistent with constructionist pedagogy; the idea that students effectively leam through the 
construction of some artifact (Papert, 1980). Programming languages such as Logo, Alice, and 
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Scratch have been used by teachers to teach computer science and other subject areas through 
the designing of games with much success in improving student performance (Kafai, Ching, & 
Marshall, 1997; Peppier & Kafai, 2007). While programming in MS PowerPoint does not require 
the same level of cognitive demand as the aforementioned languages, constructing the games 
requires some level of s ki ll (e.g., planning the sequence of slides, inserting links correctly, 
debugging the game of errors, etc.) with the software application. Proponents argue that the 
simplicity, as well as the availability of MS PowerPoint in schools, is a distinct advantage over 
using other programming languages as tools in subject areas outside of computer science 
(Barbour, Kinsella, & Rieber, 2011). The second justification for the use of homemade 
PowerPoint games is the use of narratives in the design of the game. Many games have a short, 
concise storyline that provides background as to why one is playing the game. While Bangert- 
Drowns, Hurley, and Wilkinson (2004) only found a small, positive impact in their meta-analysis 
of 48 writing across the curriculum studies, they did find enhanced effects when the length of the 
assignment was shortened. One form of writing assignment that focuses on brevity and revision 
is called microthemes (Ambron, 1987). Garner (1994) found that grades and motivation 
increased with the use of microthemes, and student surveys showed a high approval rating for the 
technique. The final justification is the students’ task of writing their own multiple-choice 
questions for the game. It is this third justification that I would like to examine in greater detail. 

For a homemade PowerPoint game to be challenging, students have to create coherent questions 
that not only contain the right answer, but also several plausible yet incorrect alternatives to 
distract the player (Rieber et al., 2008). Therefore, the student must work with the content in 
constructing questions, and even address misconceptions as they develop correct and incorrect 
choices (Chin & Osborne, 2008). Based on the review of 27 studies examining self-questioning 
techniques, Wong (1985) gave three justifications for using self-generated questions in the 
classroom. First, creating questions helped to guide students’ thinking as a form of active 
processing of content. Second, self-questioning was a metacognitive process which helps 
students gauge their own understanding. Finally, self-generated questions were supported by 
schema theory (Piaget, 1926), since the formation of questions help to integrate new information 
with current schema. Wong’s analysis of studies that used self-questioning as an instructional 
strategy found that the strategy did enhance learning, but only slightly. Examining the studies 
more closely, Wong determined that the effects were greatest when there was an emphasis on 
writing more higher-order questions, a longer processing time, and a higher amount of direct 
instruction on how to write questions. Further, Rosenshine, Meister, and Chapman (1996) 
conducted a similar study a decade later and, based on their review of 26 studies, found that 
reading comprehension generally increased when question writing was employed as an 
instructional technique. 

More recently, Lotherington and Ronda (2010) found that fourth-grade students, when creating 
online board games for a geography course, wrote more higher-order questions over time. The 
researchers also found that allowing the students to revise questions and critique the questions of 
others were important factors in the development of their question writing skills. Within the 
science discipline, Harper, Etkina, and Lin (2003) examined the benefits of the technique in an 
introductory physics course. The researchers found no correlation between the number of 
questions written by students and their test scores, but did find a significant relationship between 
the quality of the questions written and test scores. Finally, a review of student-generated 
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questioning studies in science by Chin and Osborne (2008) stressed the importance of 
scaffolding, prompts, and modeling in determining the success of the technique. 

As the bulk of the work in creating a homemade PowerPoint game consists of writing questions, 
and student-generated questioning is a generally effective strategy, it would appear that studies 
involving homemade PowerPoint games would be an effective tool to increase learning. 
However, all of the published research to date on homemade PowerPoint games has shown no 
significant difference in performance between control (i.e., students that neither constructed nor 
played games) and treatment groups. For example, Parker (2004) examined the use of homemade 
PowerPoint games to teach grammar skills to middle school students, and found no statistical 
difference between the treatment and control groups. Similarly, Barbour, Clesson, and Adams 
(2011) conducted a study in a secondary British literature class involving the use of the games 
and found no statistical difference in performance between students who made games and 
students who did not. Barbour, Kinsella, and Rieber (2011) conducted a similar study in a 
secondary U.S. history class and also found no statistically significant difference in performance. 

Siko et al. (2011) conducted the largest study to date using homemade PowerPoint games, using 
approximately 150 students enrolled in a secondary environmental chemistry course. The 
researchers not only analyzed the performance on two separate unit tests (i.e., by comparing the 
performance of those who created games and those who did not on two separate occasions), they 
examined whether creating tests on multiple occasions improved performance (i.e., if repeated 
exposure to the treatment had any effect). Similar to the previous studies, there was no 
statistically significant difference in performance on either unit test. When the researchers 
examined at the scores on the second unit test, they found that the students who created games 
for both units performed better than the students who only made the games on one occasion and 
then those who never created the games at all. However, the difference was not statistically 
significant. 

Barbour, Kromrei, et al. (2009) tested the assumption that students were writing higher-order 
questions, one of the justifications of homemade PowerPoint games, by analyzing the questions 
written by students in the Barbour, Kinsella, and Rieber (2011) study. In their analysis of over 
1,900 questions, the authors found the overwhelming majority of questions (i.e., 94%) were 
“Knowledge” level questions, which is the simplest form of question based on Bloom’s (1956) 
taxonomy. Further, none of the questions were above the “Application” level. Barbour and his 
colleagues suggested that this finding may be the reason why the studies on the effect of 
homemade PowerPoint games on student performance conducted up to that point have not 
shown statistical differences. 

In summary, while the literature supports each of the three philosophical justifications 
individually for the use of homemade PowerPoint games, studies involving the games 
themselves have not shown any statistical difference in performance. This fact has led 
researchers to begin to examine whether the games are truly demonstrating these justifications 
and, in particular, whether students are writing higher-order questions. As a follow-up to both the 
Barbour, Kromrei, et al. (2009) and Siko et al. (2011) studies, I am examining the range of 
student generated questions based on Bloom’s taxonomy when creating homemade PowerPoint 
games for an environmental chemistry course. 
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Methodology 

The purpose of this study was to analyze the questions written by students for the homemade 
PowerPoint games they created in the Siko et al. (2011) study to determine where they belonged 
on Bloom’s taxonomy. In keeping with the findings of Rosenshine et al. (1996), students write 
more higher-order questions with continued practice. Therefore, the two research questions for 
this study were as follows: 

1. How many questions from each level of Bloom’s taxonomy did students write for 
each of the two games in the Siko et al. (2011) study? 

2. Did students who created games twice write more higher-order questions than 
students who only created games once on the second unit project? 

The first research question is descriptive in nature; however, for the second question, I developed 
the following hypotheses: 

Ho: No difference in the number of questions from each level. 

Hp Students who created games on two occasions wrote more higher order questions 
than students who only created games once. 

Data Analysis 

In order to answer the first research question, I followed a protocol similar to the protocol used in 
the Barbour, Kromrei, et al. (2009) study. Two subject matter experts (i.e., teachers in the school 
used in the study who taught the course) viewed each game and then coded each question to 
determine which level on Bloom’s taxonomy the question belonged. Prior to their coding, the 
subject matter experts were given background materials on Bloom’s taxonomy as well as 
material related to the Barbour et al. (2009) study. A “Knowledge” question was one that simply 
asked the student to recall some fact presented in the content of the chapter. A “Comprehension” 
question required the student to use some combination of facts in developing the answer. Finally 
an “Application” question required students to use existing knowledge in a novel situation. 
Examples of actual student questions are listed below: 

• Knowledge: What is the relationship between frequency and wavelength? 

• Comprehension: Why are greenhouse gases in our atmosphere necessary for our 
survival? 

• Application: You have 500L of a gas at 2atm at 273K. You change the temperature to 
293K. What is the new pressure if volume stays constant? 

For each unit, the subject matter experts coded three games individually, and then compared their 
results to clarify any questions they had with the application of Bloom’s taxonomy. After 
comparing their results and rectifying any problems or questions they had, they went on to code 
the remainder of their games individually. The results from both coders were tallied by both total 
number and percentage from each level on the taxonomy; thus, the total number of questions 
listed is twice the number of actual questions written by students. Inter-rater reliability was also 


Are They Climbing the Pyramid? Rating Student-Generated Questions in a Game Design Project 


7 


CJLT/RCAT Vol. 39(1) 


calculated as a percentage of questions scored the same by both coders. Inter-rater reliability was 
85.8% for the first unit test. In an attempt to improve the inter-rater reliability, the subject matter 
experts practiced by first coding one game together, followed by coding two additional games 
individually, and then meeting to come to a consensus on the two additional games. The rest of 
the games for the second unit were then coded individually. As a result, the inter-rater reliability 
improved to 96.4% for the second unit test. 

To answer the second research question, I examined questions written by students on the game 
project for the second unit. I compared the number of knowledge-level questions written by 
students who created games twice versus students who only created games once. To test the 
hypothesis, I used an independent /-test to see if the students who created games twice wrote 
fewer knowledge-level questions. 

Participants and Setting 

The questions analyzed in the study were from games created by students at a large, Midwestern 
U.S. high school during the 2008-2009 school year. The course for which the students created 
games was entitled environmental chemistry. The course was based on the American Chemical 
Society’s Chemistry in the Community curriculum, also known as ChemCom. The ChemCom 
curriculum is different than a traditional high school chemistry course in several ways. First, the 
curriculum emphasizes the more practical aspects of chemistry that most people would see in 
everyday life (American Chemical Society, 2008). For example, instead of units on 
stoichiometry and gas laws, the ChemCom curriculum has units on water quality, petroleum, and 
air quality. Second, the course has less emphasis on both memorization and mathematic problem 
solving than a traditional chemistry course. Finally, the course is geared toward college-bound 
student who do not intend to pursue a career in science or engineering. 

The school where the games were created utilized a trimester system, with the course being two 
trimesters in length. The students did not have to have the course in successive trimesters (i.e., 
students could be enrolled during the first and second, the second and third, or the first and third 
trimesters). The first unit test occurred during the first half of the course, and the second unit test 
occurred in the second half of the course. Both tests came near the end of their respective 
trimester. Students also did not necessarily have the same teacher for both halves of the course. 
Since only one of the three teachers who taught the course during the 2008-2009 school year had 
the students create games for the class, it was possible that students created games for both units, 
for the second unit only, or not at all. The first unit that homemade PowerPoint games were 
made was on natural resources, the periodic table, mining, and processing metals. The content 
for the second unit revolved around atmospheric conditions, properties of gases, and the gas 
laws. 

For both units, the students followed a protocol that consisted of four consecutive days in the 
computer lab. On the first day, students were introduced to the project by playing sample games 
downloaded from the homemade PowerPoint game website 

( http://it.coe.uga.edu/wwild/pptgames) and discussing the attributes found in high-quality, 
interesting games. While working in groups of two or three students, they also began 
brainstorming ideas for narratives and questions. On the second and third days, the students 
developed questions for the game and started to construct the game from a template downloaded 
from the homemade PowerPoint game website. On the final day, students finished their games 
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and played the games created by other students. Shortly thereafter, the students took a test on the 
unit. 

Results 

The first research question examined how many higher order questions students wrote on each 
test. After coding 1,250 questions, the majority of the questions were judged to be knowledge 
level questions. Table 1 summarizes our results for the first unit on materials and resources. 


Table 1: Percentage of questions written rated from each level of Bloom’s taxonomy 
on the first unit test 


Level 

Number of Questions 

Percentage of Questions 

Knowledge 

760 

60.8% 

Comprehension 

285 

22.8% 

Application 

205 

16.4% 

Total 

1,250 

100.0% 


As previously mentioned, analysis of the ranking showed an 85.8% inter-rater reliability. No 
questions were ranked higher than application on Bloom’s taxonomy. 

For the second unit, where students created games on the topic of gases and the atmosphere, the 
questions were analyze by the same subject matter experts. As previously mentioned, the inter¬ 
rater reliability improved to 96.4%. The results are summarized in Table 2. 

Table 2: Percentage of questions written rated from each level of Bloom’s taxonomy 
on the second unit test 


Level 

Number of Questions 

Percentage of Questions 

Knowledge 

890 

67.4% 

Comprehension 

216 

16.3% 

Application 

216 

16.3% 

Total 

1,322 

100.0% 


Again, no questions were ranked above the level of application and the majority of the questions 
were rated as knowledge questions. 

To answer the second research question, data from the second unit were categorized based on 
whether the students created games for either both units or just the second unit. Comparisons 
were made between the percentages of knowledge questions in each game rather than total 
number of questions. There were 14 groups who created games for only the second unit, while 
16 groups created games for both units. Most groups contained two members, but several groups 
contained three, because of students being absent or an odd number of people in the class. The 
game project called for each group member to write ten questions; thus, most games consisted 20 
or 30 questions. However, some groups wrote fewer than the required number of questions, and 
other groups wrote more than the required number. Therefore, it was necessary to analyze the 
data based on percentages rather than total number. The results are summarized in Table 3. 
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Table 3: Percentage of questions for each game rated as knowledge level 


Group 

N 

Mean 

SD 

First time with games 

14 

70.7 

10.1 

Second time with games 

16 

64.6 

10.5 


Groups who only created games on one occasion wrote a higher percentage of “Knowledge” 
level questions than groups who created games for both units. In other words, the group who 
created games for both units wrote a greater percentage of higher-order questions. However, the 
difference was not determined to be statistically significant, t( 28) = 1.60; p = .12. 

Discussion 

The results of this study show that in a game design project in an environmental chemistry class, 
the questions generated by students were primarily knowledge-level questions. This was the case 
for games created for both units where the game design project was used. Further, on the second 
unit test there was no statistical difference in the percentage of knowledge-level questions 
between groups of students who created games where students created games on two occasions 
and those who only created games once. 

These results, along with the results of the study from which the data came (Siko et al., 2011) 
mirror the results of the Barbour and his colleagues (Barbour, Clesson, & Adams, 2011; Barbour, 
Kinsella, & Rieber, 2011) studies. That is, both studies showed no statistical difference in 
performance between groups who made games and those who did not, as well as the fact in both 
studies the students wrote a majority of knowledge level questions. Barbour, Kromrei, et al. 
(2009) believed that the high proportion of knowledge level questions may have been a reason 
for the no statistical difference findings. Harper, Etkina, and Lin (2003) also found that it was not 
the number of questions written, but the number of quality questions written by students that 
influenced performance. However, the deeper question becomes why the students are not writing 
more higher-order questions in the first place, and whether any of the three justifications are 
actually being met with the current protocol for a game design project using homemade 
PowerPoint games. 

One of the problems may be in the way the game projects have been conducted in the first place. 
They have been used as review for an assessment. Siko et al. (2011) questioned whether this 
actually constituted constructionism. In other words, can a review and the actual learning of the 
content be considered the same with respect to constructionism? Perhaps future studies could 
examine student performance when the game project was part of the actual content delivery, or if 
the games were constructed throughout the unit, rather than at the end. This would make a 
stronger case that the game design project is indeed constructionism. 

The literature involving writing across the curriculum and microthemes stated that repetition 
were helpful in allowing students to write better (Garner, 1994). Students are writing short 
statements for the theme, a technique supported by Bangert-Drowns et al. (2004), but for the 
most part the act of writing a narrative while creating a homemade PowerPoint game was a one- 
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time event. However, while these issues may exist with application of the homemade PowerPoint 
games and how adequately they satisfy their philosophical justifications of constructionist 
pedagogy and microtheme writing, it is doubtful that these deficiencies affected the students’ 
ability to write more higher-order questions. 

With regards to question writing strategies, Wong (1985) noted that the effects of the technique 
could be enhanced if more instruction was given on how to write questions and if an emphasis 
was placed on writing higher-order questions. Chin and Osborne (2008) also found that students 
needed sufficient instruction through prompts, scaffolding and modeling to be successful. By 
spending all of their time in the computer lab, it makes it difficult for a teacher to teach the 
technical aspects of the project, have the students be introduced to game design with an 
orientation to homemade PowerPoint games, work on constructing the game, and complete the 
project, let alone find time to provide adequate instruction on writing questions to the students. 

Furthermore, Papert (1980) believed that a key component of learning through programming was 
the aspect of debugging, or fixing errors in the program. While the current protocol seemed to 
provide adequate time for debugging the MS PowerPoint file itself, it did not allow the teacher 
time to provide feedback to the students. The researchers in the Siko et al. (2011) study noted 
this as a potential reason for their no significant difference finding. Perhaps the lack of adequate 
feedback not only led to no difference in student performance, but also led to students not having 
time to revise their questions - or even know their questions needed to be revised - to move 
them to higher levels on Bloom’s taxonomy. Lotherington and Ronda (2010) found that time to 
feedback, revision, and the ability to critique and edit the questions of classmates were important 
to the learning process. While students in the homemade PowerPoint game studies were able to 
play and provide feedback on the games as a whole, perhaps more time should be devoted to 
providing peer feedback on student questions, which are the main component of the content on 
which students are tested. 

In line with previous research on the questions written by students for review games, the 
majority of the questions were factual recall questions. Students who had previous experience 
creating games did write more higher-order questions than those without prior experience, but 
the difference was not statistically significant. Based on speculation from previous studies, this 
may be a reason for the lack of statistical significance in student performance on tests. Based on 
the research on question-writing strategies, the lack of higher-order questions written by students 
may stem from the lack of structure and time afforded to the project. In particular, there was a 
lack of instruction and instructional supports for teaching the process of writing questions. There 
was also little time allotted for feedback and revision of the questions. 

Conclusion and Implications 

In this study, I have looked at the ability of students to write quality, higher-order questions for a 
game design project involving homemade PowerPoint games. While the students did write more 
higher-order questions than a previous study involving an analysis of game questions, the 
majority of the students’ questions were still knowledge-level questions requiring only 
memorization and recall on the part of the player to succeed in the game. Furthermore, students 
who created games on multiple occasions did write more higher-order questions than students 
who only created games only once; however, the difference was not statistically significant. 
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Several recommendations for practitioners wanting to conduct a game design project can be 
suggested based on the results of this study. As Siko et al. (2011) originally noted, it may be 
better to implement the games as a unit project rather than simply a review tool. Also, 
researchers (e.g., Chin & Osborne, 2008; Lotherington & Ronda, 2010) suggested that more 
structure be provided when implementing the project. Based on the results of this study, I 
recommend that more structure be provided with respect to teaching students how to write 
questions. In particular, students will need more instruction on how to write higher-order 
questions and how to revise knowledge-level questions to increase their difficulty on Bloom’s 
taxonomy. Students should also have the opportunity to obtain feedback on their questions from 
the teacher and, if possible, have students revise one another’s questions as well. 

Future research should examine whether the aforementioned suggestions increase the number of 
higher-order questions written by students, and also whether the additional structure increases 
the performance of students who create homemade PowerPoint games. The changes in structure 
would also affect how the overall instruction is designed for the unit. Four days in a computer lab 
prior to a test is quite different than spreading that time out over the course of a unit. Students 
may receive instruction on question writing and time to write the questions in the classroom 
rather than the computer lab. From a design perspective, researchers could examine the design 
decisions made by a classroom teacher to intertwine the game project throughout the unit rather 
than at the end. In the end, the game itself would shift from a simple review tool to a driving 
question or artifact in a project-based science unit, which could erase any questions raised on 
whether homemade PowerPoint games are truly rooted in constructionist pedagogy. 
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